Inferring 3D face models from 2D databases

Parameterized Appearance Models (PAMs) such as Active Appearance Models (AAMs) or Morphable Models (MMs) are a popular tool to model the shape and appearance of faces in images. Although extensively used, PAMs have some limitations to decouple out-of-plane pose, expression and identity changes. For instance, standard AAMs use a unique basis to model pose, identity and expression changes. On the other hand, MMs make use of a 3D face model that allows to recover 3D motion parameters, and decouple pose from identity/expression changes. However, current methods to build 3D face models have several drawbacks: (1) the variability of expression instances in existing databases is limited (2) collecting the data usually involves complex 3D capture setups (3) registering the training samples is a laborious and error prone task. This paper proposes a method to learn a rich 3D face model from large collections of 2D hand-labeled databases containing many subjects under different expressions and poses. We develop an incremental Structure-from-Motion (SfM) approach to build 3D generic face models from 2D instances and a prior 3D model only containing neutral expressions. Experimental results show the effectiveness and robustness of the proposed technique to fit the learnt 3D shape model to previously unseen subjects.